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Enclosed is a copy of the prior application; including the specification, claims, drawings, oath or 
declaration showing the applicant's signature, and any amendments referred to in the oath or 
declaration filed to complete the prior application. (It is noted that no amendments referred to in the 
oath or declaration filed to complete the prior application introduced new matter therein.) The 
continuing application is as follows: 22 pages of specification, 18 claims, 1 page of abstract, 8 sheets 
of informal drawings, and 5 pages of oath or declaration. 

The entire disclosure of the prior application, from which a copy of the oath or declaration is supplied, 
is considered as being part of the disclosure of the accompanying application and is hereby 
incorporated by reference therein. 

Amend the specification by inserting before the first line the sentence: "This application is a 
Continuation of application Serial No. 08/987,91 1, filed December 9, 1997, entitled "METHOD OF, 
SYSTEM FOR, AND COMPUTER PROGRAM PRODUCT FOR PROVIDING EFFICIENT 
UTILIZATION OF MEMORY HIERARCHY THROUGH CODE RESTRUCTURING", which 
application is incorporated herein by reference." 

A prelirrhnary amendment is enclosed. (Claims added by this amendment have been properly 
numbered consecutively beginning with the number next following the highest numbered original 
claim in the prior application.) 
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PRELIMINARY AMENDMENT 



Assistant Commissioner for Patents 
Washington, D.C. 20231 

Dear Sir: 

Prior to a first Office Action, please amend the above-identified application as follows: 



IN THE CLAIMS 

Please amend claims 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 17, and 18 as follows: 



1 . (AMENDED) A method of restructuring a program comprising basic blocks for 
execution by a processor having a memory hierarchy [ comprising ] comprised of a plurality of 
levels, the method comprising the steps of: 

a) constructing a Program Execution Graph (PEG) corresponding to a first level of the 
memory hierarchy from control flow and frequency information from a profile of the program, the 
PEG comprising a weighted undirected graph comprising nodes and edges, each node representing 
a [ the ] basic blockf s ] . each [ and ] edge[ s ] representing a transfer of control between a pair[ s ] of 
the basic blocks, each of the nodes having a weight equal to the si2e of the basic block represented 
by the node, and each of the edges having a weight equal to a frequency of transition between a pair 
of basic blocks represented by a pair of nodes connected by the edge; 
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b) for the first level of the memory hierarchy, partitioning the nodes of the PEG into 
clusters, such that a sum of weights of the edges whose endpoints are in different clusters is 
rrrmimized, and such that, for any cluster, a sum of weights of the nodes in the cluster is no greater 
than an upper bound corresponding to a size of the first le vel of the memory hierarchy; and 

c) restructuring the basic blocks into contiguous code corresponding to the clusters^such 
that the basic blocks that communicate extensively with each other are on a same level of the 
memory hierarchy, in order to reduce communications betwee n the basi c blocks across the levels of 
the memory hierarchy , 

2. (AMENDED) The method of claim 1 further comprising the steps of: 

d) for a next level of the memory hierarchy, constructing a next PEG from the clusters of 
the partitioned PEG, such that a node in the next PEG corresponds to a cluster in the partitioned 
PEG, and such that there is an edge between two nodes in the next PEG if there is an edge between 
components of the clusters represented by the two nodes; and 

e) assigning a weight to each node of the next PEG; [ and ] 

f) assigning a weight to an edge between a pair of nodes of the next PEG representing a pair 
of clusters of the partitioned PEG, the edge weight being a summation of weights of edges in the 
partitioned PEG having endpoints in the pair of clusters in the partitioned PEGiand 

g^ partitioning the nodes of the next PEG into cluster s - such that a sum of weights of the 
edges whose end points are in different clusters is mini miz ed, a nd such that, for any cluster, a sum of 
weights of the nodes in the cluster is no greater t han an upper bound corresponding to a size of a 
next level of the memory hierarchy . 

3. (AMENDED) The method of claim 2 further comprising the steps of: 

h^ makin g the next PEG the PEG to be partitioned for an additional lev el of the memory 

hierarchy, if any: and 

i [ f ] ) repeating steps b through f for the additional level o f the memory hierarchy, if any. 

4. (UNCHANGED) The method of claim 1 wherein the upper bound is a multiple of a size 
of a level of the memory hierarchy. 
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5. (AMENDED) The method of claim 3 wherein the upper bound for a level of the 
memory hierarchy, other than a first level, is a size of a level of the memory [ level ] divided by an 
upper bound used to partition a next lower level of the memory hierarchy. 



6. (AMENDED) The method of claim 3 further comprising the steps of: 

removing a basic block whose size is greater than the upper bound from the partitioning 

step; and 

[ reintegrating ] including the removed basic block [ whose size is greater than the upper 
bound into ] in a next repetition of steps b through f. 

7. (AMENDED) An article of manufacture for use in a computer system for restructuring a 
program comprising basic blocks for execution by a processor having a memory hierarchy 

[ comprising] comprised of a plurality of levels, [ said ] the article of manufacture comprising a 
computer-readable storage medium having a computer program embodied in [ said ] die medium 
which may cause the computer system to: 

a) construct a Program Execution Graph (PEG) corresponding to a first level of the 
memory hierarchy from control flow and frequency information from a profile of the program, the 
PEG comprising a weighted undirected graph comprising nodes and edges, each node representing 

[ the ] a basic block[ s ] .each [ and ] edge[ s ] representing a transfer of control between a pair[ s ] of 
the basic blocks, each of the nodes having a weight equal to the size of the basic block represented 
by the node, and each of the edges having a weight equal to a frequency of transition between a pair 
of basic blocks represented by a pair of nodes connected by the edge; 

b) for the first level of the memory hierarchy, partition the nodes of the PEG into clusters, 
such that a sum of weights of the edges whose endpoints are in different clusters is minimized, and 
such that, for any cluster, a sum of weights of the nodes in the cluster is no greater than an upper 
bound corresponding to the first level of the memo ry hierarchy; and 

c) restructure the basic blocks into contiguous code corresponding to the clusters, such that 
the basic blocks that communicate extensively with each o t her are on a same level of the memory 
hierarchy, in order to reduce communications between the basic b locks across the levels of the 
memory hierarchy . 
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8. (AMENDED) The article of manufacture of claim 7 wherein the computer program may 

further cause the computer system to: 

d) for a next level of the memory hierarchy, construct a next PEG from the clusters of the 
partitioned PEG, such that a node in the next PEG corresponds to a cluster in the partitioned PEG, 
and such that there is an edge between two nodes in the next PEG if there is an edge between 
components of the clusters represented by the two nodes; [ and ] 

e) assign a weight to each node of the next PEG; [ and ] 

f) assign a weight to an edge between a pair of nodes of the next PEG representing a pair of 
clusters of the partitioned PEG, the edge weight being a summation of weights of edges in the 
partitioned PEG having endpoints in the pair of clusters in the partitioned PEGiand 

o) partitioning the nodes of the next PEG into clusters, such th at a sum of weights of the 
edges whose endpoints are in different clusters is minimized, and s uch that, for any cluster, a sum of 
weights of the nodes in the cluster is no greater than an upper bound corresponding to a size of a 
next level of the memory hierarchy . 

9. (AMENDED) The article of manufacture of claim 8 wherein the computer program may 

further cause the computer system to: 

h) make the next PEG the PEG to be partitioned for an additional level of the memory 

hierarchy, if any: and 

i [ f ] ) repeat steps b through f for the additional level of the memory hierarchy, if a ny. 

10. (UNCHANGED) The article of manufacture of claim 7 wherein the upper bound is a 
multiple of a size of a level of the memory hierarchy. 

1 1 . (AMENDED) The article of manufacture of claim [ 10] 9 wherein the upper bound for 
a level of the memory hierarchy, other than a first level, is a size of a level of the memory hierarchy 
level divided by an upper bound used to partition a next lower level of the memory hierarchy. 

12. (AMENDED) The article of manufacture of claim 9 wherein the computer program 
may further cause the computer system to: 
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remove a basic block whose size is greater than the upper bound from the partitioning step; 

and 

[ reintegrate ] include the removed basic block [ whose size is greater than an upper bound 
into ] in a next repetition of steps b through f. 

13. (AMENDED) A computer system for restructuring a program comprising basic blocks 
for execution by a processor having a memory hierarchy comprised of a plurality of levels, the 
computer system comprising: 

a) a Program Execution Graph (PEG) corresponding to a first level of the memory hierarchy 
constructed from control flow and frequency information from a profile of the program, the PEG 
comprising a weighted undirected graph comprising nodes and edges, each node representing [ the ] 
a basic block[ s ] ? each [ and ] edge[ s ] representing a transfer of control between [ pairs ] a pair of 
the basic blocks, each of the nodes having a weight equal to the size of the basic block represented 
by the node, and each of the edges having a weight equal to a frequency of transition between a pair 
of basic blocks represented by a pair of nodes connected by the edge; 

b) a partition of the nodes of the PEG into clusters for the first level of the memory 
hierarchy, such that a sum of weights of the edges whose endpoints are in different clusters is 
minimized, and such that, for any cluster, a sum of weights of the nodes in the cluster is no greater 
than an upper bound corresponding to a size of the first level of the memory hierarchy ; and 

c) a restructuring of the basic blocks into contiguous code corresponding to the clusters, 
such that the basic blocks that communicate extensively with each other are on a same level of the 
memory hierarchy, in order to reduce communications between the basic blocks across the levels of 
the memory hierarchy . 

14. (AMENDED) The computer system of claim 13 further comprising: 

d) a next PEG constructed from the clusters of the partitioned PEG for a next level of the 
memory hierarchy, such that a node in the next PEG corresponds to a cluster in the partitioned 
PEG, and such that there is an edge between two nodes in the next PEG if there is an edge between 
components of the clusters represented by the two nodes; [ and ] 

e) a weight assigned to each node of the next PEG; [ and ] 
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f) a weight assigned to an edge between a pair of nodes of the next PEG representing a pair 
of clusters of the partitioned PEG, the edge weight being a summation of weights of edges in the 
partitioned PEG having endpoints in the pair of clusters in the partitioned PEG ; and 

g) a partitioning of the nodes of the next PEG into clusters such that a sum of weights of 
the edges whose endpoints are in different clusters is minimized, and such that, for any cluster, a 
sum of weights of the nodes in the cluster is no greater than an upper bound corresponding to a 
size of a next level of the memory hierarchy . 

15. (AMENDED) The computer system of claim 14 further comprising: 

h) a conversion of the next PEG into the PEG to be partitioned for an additional level of 
the memory hierarchy, if any; and 

i [ f ] ) a repetition of elements b through f for the additional level of the memory hierarchy. 

if any . 
7- 

16. (UNCHANGED) The computer system of claim 13 wherein the upper bound is a 
multiple of a size of a level of the memory hierarchy. 

17. (AMENDED) The computer system of claim 15 wherein the upper bound for a level of 
the memory hierarchy, other than a first level, is a size of a level of the memory hierarchy level 
divided by an upper bound used to partition a next lower level of the memory hierarchy. 

18. (AMENDED) The computer system of claim 15 further comprising: 

a removal of a basic block whose size is greater than the upper bound from the partitioning 
step; and 

[ a reintegration ] an inclusion of the removed basic block [ whose size is greater than the 
upper bound into ] in a next repetition of elements b through f. 

REMARKS 

Prior to a first Office Action in this application, Applicants request that original claims 1, 2, 
3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 17, and 18 be amended. These amended claims do not involve any 
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new matter or objectionable changes. When the Examiner takes this application up for action, he is 
requested to take the foregoing into account. 

It is submitted that this application is now in good order for allowance and such allowance is 
respectively solicited. Should the Examiner believe minor matters still remain that can be resolved in a 
telephone interview, the Examiner is urged to call Applicants' undersigned attorney. 

Respectfully submitted, 

D2 Ching Ju et al. 

By their attorneys, 

GATES & COOPER 



Date: October 10. 2000 



GHG/10 



6701 Center Drive West, Suite 1050 
Los Angeles, California 90045 
(310) 641-8797 




Name: Geefrge H. Gate; 
Reg. No.: 33,500 
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SPECIFICATION 

IBM Docket No. ST9-95-010 Express Mail #: EG340732293US 

TO ALL WHOM IT MAY CONCERN: 

;fBE IT KNOWN that We, Dz Ching Ju of Sunnyvale, California and citizen of the Taiwan, Kalyan 
f Muthukumar of Cupertino, California and citizen of India, Shankar Ramaswaray of Bethel Park, 
ri Pennsylvania, and citizen of India, and Barbara Bluestein Simons of Palo Alto, California, and citizen 
% of the United States, have invented new and useful improvements in 



A METHOD OF, SYSTEM FOR, AND COMPUTER PROGRAM PRODUCT FOR 
PROVIDING EFFICIENT UTILIZATION OF MEMORY HIERARCHY 

THROUGH CODE RESTRUCTURING 

of which the following is a specification: 
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1 TITLE OF THE INVENTION 

2 

3 A METHOD OF, SYSTEM FOR, AND COMPUTER PROGRAM PRODUCT FOR 

4 PROVIDING EFFICIENT UTILIZATION OF MEMORY HIERARCHY 

5 THROUGH CODE RESTRUCTURING 

6 
7 
8 

9 O 

0 m 

1 U CROSS REFERENCES TO RELATED APPLICATIONS 

2 J 

3 |I ■ 

14 L 

16 O 

1 7 a STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY 

1 8 SPONSORED RESEARCH AND DEVELOPMENT 

19 
20 

21 _ 

22 

23 

24 A portion of the Disclosure of this patent document contains material which is subject to 

25 copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone 

26 of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office 

27 patent file or records, but otherwise reserves all copyright rights whatsoever. 
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1 BACKGROUND OF THE INVENTION 

2 
3 

4 1. Field of the Invention 

5 

6 This invention relates generally to optimizing compilers for development of computer 

7 programs for use on a computer, and more particularly to code restructuring. 
8 

9 O 

10 cp 2. Description of the Related Art 

11 s - 

•> - , H 
S " 

1 2 *F Advances in processor designs have provided very high speed instruction processing. In order 

1 3 | - to sustain this speed, processors need very effective usage of the memory hierarchy of the machines. 

14 y T Ensuring effective usage is difficult for large programs like compilers or database engine codes that 

15 ■ 1 have complex control flow. This occurs, for example, as a result of having to consider inputs of 

16 □ different types or handle error conditions appropriately. Consequently, code generated for large 

1 7 n complex programs consists of sections that are heavily executed intermingled with sections that are 

1 8 infrequently executed, resulting in poor utilization of the memory hierarchy. Code restructuring may 

1 9 improve memory hierarchy utilization and, consequently, performance. 
20 

2 1 The state of the art of code restructuring techniques is described in Simons (B. Simons, "Code 

22 Positioning for Procedures and Basic Blocks, 1 ' IBM Technical Report ADTI- 1994-006; also available 

23 as IBM Santa Teresa Laboratory Technical Report TR 03 .580.). Two representative techniques 

24 described by Simons include the IBM Heatshrink and the IBM AOPT products. Heatshrink, a trace- 

25 directed program restructuring (TDPR) tool, was designed and implemented by Randy R. Heisch. A 

26 more detailed description of Heatshrink is given in Heisch (R. R. Heisch, 'Trace-Directed Program 

27 Restructuring for AIX Executables", IBM Journal of Research and Development, vol. 38, September 

28 1994.). AOPT, a stand-alone tool developed by Itai Nahshon, attempts to locate a "hot path" in the 

29 code and then move the entire path as a single entity, rather than move individual basic blocks. 
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1 Profile information, used by both Heatshrink and AOPT, is obtained by AOPT. Profiling involves 

2 running code using sample or representative inputs. Based on the behavior of the code on the given 

3 inputs, one makes assumption about its behavior on all inputs. Heatshrink and AOPT both perform 

4 global reordering of basic blocks and are bundled together as part of FDPR (Feedback Directed 

5 Program Restructuring tool), an IBM Licensed Program Product for IBM AIX Operating System, 

6 Version 4.1. 
7 

8 Pettis et al. (K. Pettis and R. C. Hansen, "Profile Guided Code Positioning," Proceedings of 

9 the ACM SIGPI^AN '90 Conference on Programming language Design and Implementation (PLDI) % 

10 r pp. 16-27, June 1990.) follow a different approach, that also relies on profiling information, but that 

1 1 r moves executed basic blocks only within a procedure in addition to moving procedures. Basic blocks 

12 E that are not executed according to the profile data are moved to a separate region at the end of an 

13 ordering. 
14 

1 5 These and other conventional techniques for code restructuring attempt to keep frequently 

16 y executed code close together in memory. Intuitively, such an approach tries to retain the working set 

1 7 □ of frequently executed code in cache. If the working set fits in the cache, this approach is reasonable; 

1 8 on the other hand, if the cache is small or if the effective portion of the cache available to an 

1 9 application is small due to multiprogramming, conventional code restructuring techniques may not 

20 perform well. A major flaw of these conventional approaches is that "closeness" in memory does not 

21 actually relate to cache or paging performance. Two basic blocks may be "close" together in 

22 memory, and yet there may still be a cache miss or page fault transitioning between the two basic 

23 blocks, or they may be very far apart and still cause the same effect of a cache miss or page fault. 

24 Thus, conventional code restructuring techniques fail to realize that blocks interacting frequently 

25 should be kept in the same cache line or page rather than "close" to each other in memory. 

26 Conventional code restructuring techniques also fail to realize that memory hierarchy hardware 

27 parameters such as cache line size, page size, cache associativity size and translation lookaside buffer 

28 (TLB) size may be considered in determining the code structure of an application. 
29 
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1 Thus, the conventional code restructuring techniques fail to consider memory hierarchy. As 

2 such, there is a need for a method of, system for, and computer program product for, providing code 

3 restructuring dependent upon a memory hierarchy. 
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I SUMMARY OF THE INVENTION 

2 

3 The invention disclosed herein comprises a method of, a system for, and an article of 

4 manufacture for providing code restructuring based on profiling information and memory hierarchy, 

5 that is also scalable to arbitrary levels of memory hierarchy. 
6 

7 The present invention reorders code to reduce communications across cache line and across 

8 page boundaries. The present invention also reorders code to reduce the likelihood that chunks of 

9 . code that communicate with each other and that have the same cache or TLB associativity set will be 
1 0 m assigned to different cache lines or pages. The present invention may be generalized to deal with 

I I H multiple levels of cache or other types of memory hierarchy. 

12 £ 

13 The present invention accomplishes this by reordering code such that chunks of code that 

14 communicate extensively with each other are on the same cache line, page, or other level of memory 

1 5 - hierarchy. Reordering is performed by first clustering the nodes, performing first level padding of 

1 6 b unfilled clusters with basic blocks that were not accessed in profiling, and then performing additional 

1 7 t padding using no-operation instructions (NOP or NO-OP) to fill out any unfilled portion of that level 

1 8 of the memory hierarchy. For example, to reorder code for a first level cache line, nodes are clustered 

1 9 to reduce communications across cache lines, any of these clusters which are unfilled clusters are 

20 padded with basic blocks that were not accessed in profiling, and then any unfilled cache lines are 

2 1 further padded with NOPs to fill out the cache lines. _ 
22 

23 The present invention avoids set collisions by considering both memory hierarchy size and 

24 associativity. Associativity means that there is a connection between a memory location and the place 

25 in the memory hierarchy to which the data for that memory location is assigned. 4-way associativity, 

26 for example, means that the cache is divided into four sets, and each memory location is uniquely 

27 mapped into one of those four sets. This mapping may be done by using two bits of the memory 

28 location address. If a cache line is 128 bytes in size, and if a memory word contains 4 bytes, then 32 

29 words will fit on one cache line. A primary objective of the associativity mapping is to have each 
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1 consecutive chunk of 32 words map to different associativity sets. If bits 6 and 7 of the memory 

2 location address are used to determine to which associativity set a 32 bit chunk maps, then adjacent 

3 32 word chunks map to different sets. A set collision occurs when all the cache lines in a particular 

4 set are full. This may occur even if one of the other sets has space. As conventional processor 

5 implementations hard wire the associativity between memory and cache sets, data may not arbitrarily 

6 occupy a cache line in an associativity set that differs from the one to which the particular memory is 

7 assigned. The present invention considers both memory hierarchy size and associativity to avoid such 

8 associativity set collisions by dividing the cache size by its associativity, i.e., if an 8K cache is 4-way 

9 O set associative, then n2 = 2048 bytes. This n2 falls between the cache line size, n 1 , of 1 28 bytes and 

10 i the page size, n3, of 4096 bytes. Similarly, the TLB size divided by its associativity is considered. 

1 1 f ? Alternative embodiments of the present invention may be modified to ignore various levels of the 

1 2 i memory hierarchy; for example, an alternative embodiment may ignore associativity. 

13 p 

14 f The present invention restructures code by constructing a Program Execution Graph (PEG) 

1 5 O corresponding to a level of the memory hierarchy, partitioning this PEG to reduce estimated memory 

1 6 5 overhead costs, and constructing a PEG for a next level of the memory hierarchy from the partitioned 

17 J=;: PEG. The present invention may be performed by: 
18 

1 9 a. constructing a Program Execution Graph (PEG) from control flow and frequency information 

20 from a profile of the program to be restructured. The profile of the program is produced by 

21 running the program using sample or representative inputs. The present invention assumes 

22 that the profiled program behavior is representative of the general program behavior, i.e., that 

23 basic blocks that communicate in the profiling will communicate in general. This type of 

24 profiling assumption is commonly used in practice by those skilled in the art. The PEG is a 

25 weighted undirected graph comprising nodes representing basic blocks and edges representing 

26 transfer of control between pairs of basic blocks. The weight of a node is the number of bytes 

27 in the basic block it represents, and the weight of an edge is the frequency of transition in the 

28 profiling between the pair of basic blocks it connects. Those skilled in the art recognize that 

29 the unit of measurement for the weight may be a unit other than bytes, i. e., bits, words, etc. 
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1 b. partitioning or clustering the nodes of the PEG into clusters such that the sum of the weights 

2 of the nodes in any cluster is no greater than an upper bound, and the sum of the weights of 

3 the edges between the clusters is minimized. 
4 

5 c. constructing a next PEG from the clusters of the partitioned PEG such that a node in the next 

6 PEG corresponds to a cluster in the partitioned PEG, and such that there is an edge between 

7 two nodes in the next PEG if there is an edge between the clusters represented by the two 

8 nodes. 
9 

10 y d. assigning weights to the nodes and edges of the next PEG to produce a PEG. 

1 2 e. repeating steps b through e for each level of the memory hierarchy. 

13 U 

14 £ restructuring the basic blocks corresponding to the clusters into contiguous code starting at an 

1 5 appropriate memory boundary, and adding NOPs as needed so that the number of memory 

16 locations occupied by each cluster is equal to an upper bound of the cluster weight. 

1 7 

1 8 In accordance with one aspect of this invention, a Program Execution Graph constructor 

1 9 constructs a Program Execution Graph (PEG), a weighted undirected graph comprising nodes 

20 representing basic blocks or clusters of basic blocks and edges representing transfer of control 

21 between pairs of basic blocks where each basic block in the pair is in a different cluster, wherein a 

22 weight of a node of the PEG is a size of a represented basic block, and a weight of an edge is a 

23 frequency of transition between a pair of basic blocks that the edge connects or a frequency of 

24 transition between elements of two clusters that the edge connects. 
25 

26 In accordance with another aspect of this invention, the PEG, a weighted undirected graph G 

27 = (V t E) with edge and node weights, is partitioned into clusters such that the sum of the weights of 

28 the edges whose endpoints are in different partitions is minimized and the sum of weights of nodes in 

29 each set is no greater than a given upper bound. 
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1 In accordance with another aspect of this invention, a PEG for level / of a memory hierarchy, 

2 q = (V p EJ, is partitioned into clusters, and these clusters from level / are the nodes for G, + , = (V„„ 

3 £, + ^, a PEG for level /'+ / of the memory hierarchy. 
4 

5 In accordance with another aspect of this invention, if u and v e V^„ there is edge (u,v) 

6 between u and v if and only if there is an edge between some pair of nodes in G„ one of which is in 

7 the cluster corresponding to u and one of which is in cluster corresponding to v. 
S 

9 In accordance with another aspect of this invention, the weight of edge (w,v) in G, +/ is the sum 

1 0 11 of the weights of the edges between component nodes of « and v in G,. 

1 1 iii 

1 2 H. In accordance with another aspect of this invention, the upper bound for a level of the 

1 3 ' memory hierarchy may be equal to the size of that level of the memory hierarchy. 

14 M. 

1 5 J In accordance with another aspect of this invention, the upper bound for a level of the 

1 6 f memory hierarchy may be equal to a multiple of the size of that level of the memory hierarchy. 

17 O 

j g T he present invention has the advantage of restructuring code responsive to a memory 

1 9 hierarchy. 

20 

21 The present invention has the further advantage of reducing memory hierarchy access cost of 

22 restructured code. 
23 

24 The present invention has the further advantage of reducing cache misses between basic 

25 blocks of restructured code. 
26 

27 The present invention has the further advantage of reducing page faults between basic blocks 

28 of restructured code. 
29 
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1 The present invention has the further advantage of restructuring code responsive to multiple 

2 levels of cache or other types of memory hierarchy. 
3 

4 The present invention has the further advantage of being extendable to deal with additional 

5 memory hierarchies, such as second level (L2) or higher caches that may be introduced in future 

6 machines. 
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1 BRIEF DESCRIPTION OF THE DRAWINGS 

2 

3 For a more complete understanding of the present invention and the advantages thereof 

4 reference is now made to the Detailed Description in conjunction with the attached Drawings, in 

5 which: 
6 

7 Figure 1 shows a Program Execution Graph G, of a first memory hierarchy level; 

8 

9 Figure 2 shows a partitioned Program Execution Graph G t produced in accordance with the 

1 0 0 present invention; 

1 1 i 

1 2 Figure 3 shows a Program Execution Graph G 2 of a second memory hierarchy level produced 

1 3 from the clusters of the partitioned Program Execution Graph G, in accordance with the present 

14 h invention; 

15 

1 6 Figure 4 shows a partitioned Program Execution Graph G 2 produced in accordance with the 

17 □ present invention; 
18 

1 9 Figure 5 shows a Program Execution Graph G s of a third memory hierarchy level produced 

20 from the clusters of the partitioned Program Execution Graph G 2 in accordance with the present 

21 invention; — 
22 

23 Figure 6 and Figure 7 are flowcharts illustrating the operations preferred in carrying out the 

24 present invention; and 
25 

26 Figure 8 shows a computer system used in performing the method of the present invention, 

27 forming part of the apparatus of the present invention, and which may use the article of manufacture 

28 comprising a computer-readable storage medium having a computer program embodied in said 

29 medium which causes the computer system to practice the present invention. 
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I DESCRIPTION OF THE PREFERRED EMBODIMENTS 

2 
3 

4 Referring now to Figure 1 through Figure 5, sample Program Execution Graphs are shown 

5 to illustrate the practice of the present invention. The control flow and frequency information of the 

6 profiled program is organized in the form of a Program Execution Graph (PEG) which is a weighted 

7 undirected graph. For example, Figure 1 illustrates a PEG G„ generally referred to as 101, where G, 

8 = ( V„ E,). A node in the set of nodes V, corresponds to a basic block that has been executed at least 

9 = once during profiling. For G, 101, the set of nodes V, comprises the set {A 103, B 105, C 107, D 
10 1 109, E 111, F 113, G 115, H 117, 1 119, J 121, K 123, L 125, M 127, N 129, 0 131, P 133, and Q 

I I M 135}. An edge («,v), in the set of edges £„ between nodes u and v implies that there is some 

1 2 IE transition that occurs during profiling between the two basic blocks represented by « and v. For G, 

1 3 5 100, the set of nodes E, comprises the set {(A, B) 137, (B, C) 139, (C, F) 141, (F, K) 143, (K, L) 

14 145, (L, N) 147, (N, O) 149, (O, P) 151, (O, Q) 153, (B, N) 155, (F, J) 157, (J, K) 159, (B, E) 161, 

15 5 (E, F) 163, (B, D) 165, (D, E) 167, (E, G) 169, (G, I) 171, (I, L) 173, (G, H) 175, (H, I) 177, (H, M) 

16 = 179, (M, N) 181, and (D, M) 183}. The weight of a node v, written W,(v), is the size (in bytes) of 

1 7 | the basic block that v represents. In G, 101, node A 103 has a weight W,(A) of 1 14 bytes, node B 

1 g 105 has a weight W,{B) of 36 bytes, C 107 has a weight W&Q of 8 bytes, and so on. The weight of 

1 9 edge e = (u, v), written W,{e\ is the total number of transitions from basic block u to basic block v 

20 plus the total number of transitions from basic block v to basic block u. In G, 101, edge (A, B) 137 

21 has a weight W,((K B)) of 556 transitions, edge (B,C) 139 has a weight W,((B,Q) of 745 transitions, 

22 edge (C, F) 141 has a weight W,((C, F)) of 745 transitions, and so on. 
23 

24 Using the initial PEG G, for a program and given all the parameters for a n-level memory 

25 hierarchy, the present invention performs code restructuring in n or fewer steps. The i' h step 

26 constructs a new PEG by clustering PEG G, wherein graph partitioning is performed on G, to 

27 produce the clustering. Clustering and graph partitioning are discussed in the following paragraphs. 

28 
29 
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1 A cluster , written CI, is defined to be a set of nodes. A clustering of a graph G with upper 

2 bound M written C/?/s/(G,A0, is a partitioning of the nodes in G into clusters so that the sum of the 

3 node weights of the nodes in any cluster is no greater than N. Each cluster in a partitioned PEG 

4 represents a unit in the level of the hierarchy for which clustering is performed. For example, in the 

5 first stage, a unit represents the size of a cache line (in bytes) or some integral multiple of this size (to 

6 allow for basic blocks that are larger than a cache line). 
7 

8 The goal of a graph partitioning heuristic is to create clusters of nodes from the PEG such that 

9 the sum of the weights of the edges that have endpoints in different clusters is minimized, subject to 

10 the total weight of the nodes in each cluster not exceeding N. This goal is modeled by using a cost 

1 1 5 function C0. For edge e, e has zero cost in ClustfG.N), written C(e) = 0, if both endpoints of e are in 

12 C the same cluster for Clust(G,N). Otherwise, e has a cost in CIust(G,N)^ C(e) = 1 . Those skilled in the 

1 3 art recognize that costs other than zero or one may be assigned by the cost function for the purposes 

1 4 of partitioning the graph. The goal is to compute a clustering that minimizes the sum of C(e) times 

1 5 W(e) for all edges e in E. The result of graph partitioning is to create the clusters, i.e., to assign 

16 • . values to C(e) for all edges e in £. If C(e) = 0, then the endpoints of edge e are in the same cluster; 

17 :;f otherwise, the endpoints of edge e are in different clusters. 
18 

1 9 Each cluster represents a unit in the stage of the hierarchy for which clustering is performed. 

20 Because a basic block may be larger than a cache line, the first stage of the heuristic allows a cluster 

21 to be the size of some multiple of the number of bytes in a cache line. At each subsequent stage, the 

22 clusters are limited to the size of the portion of the memory hierarchy being considered, for example, 

23 a memory page size. However, if for some reason this cluster size is not preferred, the heuristic may 

24 be easily modified to assign a different value to the cluster size. 
25 

26 The cost function is an approximation of the memory hierarchy overhead incurred by the 

27 restructured program on a real machine. Formally, the code restructuring problem may be defined as 

28 a first minimization of 4> h then a minimization of then a minimization of ... , and a final 

29 minimization of & h where k is the number levels in the memory hierarchy, and where: 
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1 <p, = £ * »re; 

2 e e E, 
3 

4 The objective is to construct a clustering Cltat(G,N) so that <P, is the summation of the product of 

5 C(e) and W(e) for all e in £ f ; where C(e) = 0 if both endpoints of e are in the same cluster and C(e) = 

6 / otherwise; where the sum of the node weights of the nodes in each cluster in CluslfG^N) is no 

7 greater than N n and where is some multiple of the cache line size. 
8 

9 f For example, if a cache line size is 1 28 bytes, N, may be set to 1 28 bytes, and G, may be 

1 0 I partitioned into clusters such that the sum of the node weights of the nodes in each cluster is no 

11 o greater than 1 28 bytes where N, is a multiple of one times the cache line size. Such a partitioning of 

12 ! G, yields a clustering ClustfG^NJ = {{A 103}, {B 105, C 107, F 113}, {J 121}, {K 123, L 125}, {D 

13 2 109, E 111}, {G 115, H 117, 1 119}, {M 127, N 129}, {O 131, P 133, Q 135}}. Clustering 

14 3 Clusl(G„N,) is illustrated in Figure 2 where cluster 205 comprising {A 103} has a node weight sum 

1 5 E of 1 1 4 bytes; cluster 210 comprising {B 105, C 107, F 1 13 } has a node weight sum of 1 26 bytes; 

1 6 J cluster 215 comprising {J 121 } has a node weight sum of 92 bytes; cluster 220 comprising {K 123, L 

1 7 SS 125} has a node weight sum of 94 bytes; cluster 225 comprising {D 109, E 1 11 } has a node weight 

18 sum of 86 bytes; cluster 230 comprising {G 115, H 117, 1 119} has a node weight sum of 120 bytes; 

19 cluster 235 comprising {M 127, N 129} has a node weight sum of 80 bytes; and cluster 240 

20 comprising {O 131, P 133, Q 135} has a node weight sum of 104 bytes. 

21 — 

22 Once a clustering on G, is constructed, "padding" is performed by adding basic blocks that 

23 were not executed during the profiling to clusters that have capacity, i.e. where the sum of the 

24 weights of the nodes in the cluster is less than N, and where adding the un-executed basic block will 

25 not cause the sum of the cluster's weights to exceed N,. If there are still basic blocks remaining that 

26 have not been added to any of the clusters, and if there is no more cluster capacity to absorb these 

27 basic blocks, then they are grouped into additional clusters of their own. The technique of adding the 

28 un-executed basic block nodes to the clusters or of putting them into their own clusters is called bin 

29 packing. A greedy bin packing heuristic may be used for performing the bin backing. 
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1 Although the example of Figure 2 does not contain any un-executed basic blocks that may be 

2 used for padding (as the initial PEG does not include un-executed basic blocks in its node set), if 

3 there is an additional unexecuted basic block whose size is less than 48 bytes, it may be padded into 

4 the cluster 235 with node M 127 and node N 129 yielding a padded cluster whose sum of node 

5 weights does not exceed N, = 128. In a similar manner, a greedy bin packing heuristic may pad the 

6 other clusters. 
7 

8 Once all of the basic blocks have been put into some cluster, then the next stage is performed 

9 q which is the construction of a graph G 2 = (V y E 2 ). Each node in V 2 corresponds to a cluster in G h 

1 0 J ?; and each node in V, has a weight of one. This weight corresponds to the assignment oiN, bytes to 

1 1 =5 the basic blocks in each cluster. An edge e - (u,v) exists in E : if and only if both u and v are in V 2 , u 

12 - and v correspond to different clusters, and there is some edge in E, from one of the nodes in the 

1 3 u cluster represented by u to some node in the cluster represented by v, or visa versa. The weight of e 

1 4 is the sum of the weights of all edges in E, that have endpoints in the two clusters (u and v). 

15 O 

1 6 Referring next to Figure 3, a graph G 2 = (V* EJ constructed from the clusters Clust(G„N,) 

17 S ofG, is illustrated. G 2 is constructed by creating a node corresponding to each cluster of 

1 8 Clusl(G u NJ such that node A 305 corresponds to cluster 205, node BCF 310 corresponds to cluster 

1 9 210, node J 315 corresponds to cluster 215, node KL 320 corresponds to cluster 220, node DE 325 

20 corresponds to cluster 225, node GHI 330 corresponds to cluster 230, node MN 335 corresponds to 

21 cluster 235, and node OPQ 340 corresponds to cluster 240. Each of the nodes V 2 = {A SOSrBCF 

22 31 0, J 315, KL 320, DE 325, GHI 330, MN 335, OPQ 340 } is assigned a weight of one. Edges E 2 

23 are then constructed between various nodes V 2 where an edge e = (u, v) exists in E 2 if and only if both 

24 « and v are in V 2 , u and v correspond to different clusters, and there is some edge in E, from one of 

25 the nodes in the cluster represented by u to some node in the cluster represented by v, or visa versa. 

26 For example, edge (BCF, DE) 350 is created as there is an edge (B, D) 165 between node B 105 and 

27 node D 109, and as node B 105 and node D 109 are in different clusters, node B 105 being in cluster 

28 210 and node D 109 being in cluster 225. In a similar manner, edges (A, BCF) 345, (BCF, KL) 355, 

29 (BCF, J) 365, (BCF, MN) 370, (DE, MN) 375, (DE, GHI) 380, (GHI, MN) 385, (GHI, KL) 390, (J, 
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1 KL) 392, (KL, MN) 394, and (MN, OPQ) 396 are created. Weights are then assigned to the edges of 

2 E 2 where the weight of an edge (w, v) in E 2 is the sum of the weights of all edges in E, that have 

3 endpoints in the two clusters u and v. For example, edge (BCF, DE) 350 is assigned a weight of 

4 440, the sum of the edge weights of edge (B, D) 165, edge (B, E) 161, and edge (E, F) 163, all of 

5 these edges having endpoints in cluster 210 and cluster 225. In a similar manner, edge weights of 

6 556, 700, 750, 88, 200, 360, 800, 1 12, 128, 470, and 500, respectively, are assigned to edges (A, 

7 BCF) 345, (BCF, KL) 355, (BCF, J) 365, (BCF, MN) 370, (DE, MN) 375, (DE, GHI) 380, (GHI, 

8 MN) 385, (GHI, KL) 390, (J, KL) 392, (KL, MN) 394, and (MN, OPQ) 396. 

I o Let /; , be the size of the next level of the memory hierarchy, typically the page size in bytes. 

II i Set = nJN,. Then the same clustering heuristic is applied to G^ using the same cost function C, 

12 i.e. counting only those edges that have end points in different clusters in G 2 . This partitioning of G, 

1 3 is used to construct graph G 3 = (N* EJ by making each node in V 3 correspond to a cluster in V* and 

1 4 m having an edge in E 3 if and only if there is some edge in E 2 that has end points in nodes in the two 

15 Q clusters represented by the endpoints of the E 3 edge. Again the nodes have a weight of one, and the 

1 6 i edges have weights corresponding to the sum of the weights on the edges in G 2 that run between 

17 O some node in cluster u and some node in cluster v. 
18 

19 For example, if n the size of the next level of the memory hierarchy, is 5 1 2 bytes, 

20 then set N, = nJN, = 512/128 = 4. G, may be partitioned into clusters such that the sum of the node 

2 1 weights of the nodes in each cluster is no greater than 4. Such a partitioning of G 2 yields a clustering 

22 C1ust(G*NJ = { ABCFJKL 410 and DEGHTMNOPQ 420 } as illustrated in Figure 4. A graph G 3 - 

23 (V* EJ may be constructed from the clusters ClustfGyNJ of G 2 as illustrated in Figure 5. G 3 is 

24 constructed by creating a node corresponding to each cluster of CluslfGyN-) such that node 

25 ABCFJKL 510 corresponds to cluster 410, and node DEGHTMNOPQ 520 corresponds to cluster 

26 420. Each of the nodes V 3 = { ABCFJKL 510, DEGHTMNOPQ 520 } is assigned a weight of one. 

27 Edges E 3 are then constructed between various nodes V 3 where an edge e = (u,v) exists in E 3 if and 

28 only if both u and v are in V 3 , u and v correspond to different clusters, and there is some edge in E 2 

29 from one of the nodes in the cluster represented by u to some node in the cluster represented by v, or 
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1 visa versa. For example, edge (ABCFJKL, DEGHIMNOPQ) 530 is created as there is an edge (BCF, 

2 DE) 350 between node BCF 310 and node DE 325 which are in different clusters, node BCF 310 

3 being in cluster 410 and node DE 325 being in cluster 420. Weights are then assigned to the edges 

4 of £j where the weight of an edge (», v) in E 3 is the sum of the weights of all edges in E 7 that have 

5 endpoints in the two clusters u and v. For example, edge (ABCFJKL, DEGHIMNOPQ) 530 is 

6 assigned a weight of 1 1 1 0, the sum of the edge weights of edge (BCF, DE) 350, edge (KL, GUI) 

7 390, edge (KL, MN) 470, and edge (BCF, MN) 370, all of these edges having endpoints in cluster 

8 510 and cluster 520. 

9 Q 

10 This process of graph partitioning may be repeated for each level of the memory hierarchy for 

1 1 K j> 1. For level / of the memory hierarchy, let n, be the size of the memory hierarchy, and set N, = 

1 2 jz n/N„,. Then the same clustering heuristic is applied to G h using the same cost function C, i.e. 

1 3 f counting only those edges that have end points in different clusters in G,. This partitioning ofG, is 

14 used to construct graph G, +/ = (N i+I . E,J by making each node in V t +, correspond to a cluster in V h 

1 5 5 and having an edge in £, + , if and only if there is some edge in E, that has end points in nodes in the 

16 K two clusters represented by the endpoints of the £, +/ edge. Again the nodes have a weight of one, 

1 7 5 and the edges have weights corresponding to the sum of the weights on the edges in G, that run 

1 8 between some node in cluster » and some node in cluster v. 
19 

20 One of the advantages of the present invention is that it may be generalized to deal with 

2 1 memory hierarchies with an arbitrary number of levels. In particular, the present invention-may be 

22 applied to future improvements to memory hierarchy hardware. 
23 

24 Another advantage of the present invention is that it may be modified to deal with special 

25 cases. An example of such a special case occurs when a pair of basic blocks need to be close in 

26 memory. In this case, the edge connecting the two basic blocks may be given an exceedingly large 

27 weight, thereby forcing the two basic blocks into the same cluster if together they don't exceed the 

28 upper bound on the cluster size. This approach may also be generalized to the case in which several 

29 basic blocks are required to be close together in memory. 
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1 The present invention may also be modified to deal with the case of a program with a few very 

2 large basic blocks. Assume basic blocks B h B p and B k have weights larger than the size of the cache 

3 line, and that the weights of all the other basic blocks are no greater than the cache line size. Define 

4 G', = G, - {B„ Bj, BJ, i.e. the PEG that is obtained by removing B n B p and B k from the problem input. 

5 Let = the cache line size, and let Clusi(G'„ NJ be the clustering that is computed by the heuristic. 

6 Clusl(G'„ NJ + {B r B f BJ may then be used as the input node set to the next stage of the heuristic. 
7 

8 As the nodes representing B n B p and B k have weights greater than the value used for N , in the 

9 3 first stage of the partitioning, the weight of the node in V 2 that corresponds to is p n where p, is the 

1 0 i smallest integer such that p, * N, >= the number of bytes in B,. Values p } and p k for B y and B h 

1 1 =t respectively, are similarly computed. The heuristic may then be applied to G_„ with the above 

12 = modifications. Note that this assumes that none of B„ B r and B k is larger in bytes than n 2 . If that is 

1 3 = not the case, then again the very large basic block(s) may be removed from consideration and 

1 4 \ reintegrated at the next stage of the heuristic. 

1 5 f ; : 

j 6 q Referring next to Figure 6 and Figure 7, flowcharts illustrating operations preferred in 

1 7 } carrying out the present invention are shown. In the flowcharts, the graphical conventions of a 

1 8 diamond for a test or decision and a rectangle for a process or function are used. These conventions 

1 9 are well understood by those skilled in the art, and the flowcharts are sufficient to enable one of 

20 ordinary skill in the art to write code in any suitable computer programming language. 

21 • - 

22 Referring now to Figure 6, the process of the invention, generally referred to as 600, begins 

23 at process block 610. Thereafter, process block 615 determines memory hierarchy parameters such 

24 as the number of levels in the memory hierarchy and the size of each such level, and process block 

25 620 profiles the target program. The profiler gathers information comprising identification of basic 

26 blocks, control flow between basic blocks, and frequencies of transition between basic blocks. The 

27 present invention assumes a program is divided into basic blocks, where a basic block is a contiguous 

28 sequence of instructions with a single entry into and a single exit out of this sequence. In addition, 

29 the present invention assumes information about the transfer of control between basic blocks and the 
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1 frequency with which the transfer occurs during actual program execution. This information may be 

2 made available through the use of profiling techniques well known to those skilled in the art such as 

3 the teachings summarized in Simons (B. Simons, "Code Positioning for Procedures and Basic 

4 Blocks/ IBM Technical Report ADTI- 1994-006; also available as IBM Santa Teresa Laboratory 

5 Technical Report TR 03.580.). Responsive to the profiler provided information, process block 625, a 

6 Program Execution Graph constructor, constructs a Program Execution Graph (PEG), a weighted 

7 undirected graph comprising nodes representing basic blocks and edges representing transfer of 

8 control between pairs of basic blocks. In this PEG produced by the Program Execution Graph 

9 _ constructor, a weight of a node of the PEG is a size of a represented basic block, and a weight of an 

1 0 edge is a frequency of transition in the profiling between a pair of basic blocks that the edge connects. 

1 1 S Process block 630 then initializes a memory hierarchy level / to one, and sets /?, equal to a smallest 

12 ' power of 2 that is a multiple of the cache line size and that is greater than the sizes of all basic block 

1 3 sizes. Thereafter, process block 635 applies a heuristic H to partition G, into ClustfG^NJ, a 

14 partitioning of the nodes in G, into clusters so that the sum of the node weights of the nodes in any 

1 5 =i cluster is no greater than N,. The heuristic H solves the relaxed graph partitioning problem, and such 

16 ?! heuristics are well known to those skilled in art such as the teachings of Barnard et al. (S. T.Barnard 

1 7 G and H. D. Simon, "Fast Multilevel Implementation of Recursive Spectral Bisection for Partitioning 

18 Unstructured Problems," Concurrency: Practice and Experience, vol.6(2), pp.101-1 17, April 1994.)- 

1 9 After clustering is performed on G„ process block 640 pads the clusters with unexecuted nodes by 

20 adding basic blocks that were not executed during the profiling to clusters that have capacity, i.e. 

2 1 where the sum of the weights of the nodes in the cluster is less than N, and where adding the. 

22 un-executed basic block will not cause the sum of the cluster's weights to exceed N,. This padding is 

23 performed to reduce unused memory. If there are still basic blocks remaining that have not been 

24 added to any of the clusters, and if there is no more cluster capacity to absorb these basic blocks, then 

25 process block 640 groups these unexecuted nodes into additional clusters of their own. The 

26 technique of adding the un-executed basic block nodes to the clusters or of putting them into their 

27 own clusters is called bin packing. A greedy bin packing heuristic, which is well known to those 

28 skilled in art such as the teachings of Garey et al. (Garey, Michael and Johnson, David S. Computers 

29 and Intractability, A Guide to the Theory of NP Completeness, pages 124-127, W. H. Freeman & 
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1 Co., San Francisco, Ca., 1979) may be used to perform the bin backing. Process block 645 then 

2 creates the next level PEG G, from the level 1 partitioned PEG Clusl(G ,,N ,) . In this graph G, = (V* 

3 EJ constructed by process block 645, each node in V 2 corresponds to a cluster in G,, and each node 

4 in V 2 has a weight of one. This weight corresponds to the assignment of N, bytes to the basic blocks 

5 in each cluster. An edge e = (u, v) exists in E 2 if and only if both u and v are in u and v correspond 

6 to different clusters, and there is some edge in E, from one of the nodes in the cluster represented by 

7 u to some node in the cluster represented by v, or visa versa. The weight of e is the sum of the 

8 weights of all edges in E, that have endpoints in the two clusters (ir and v). 

9 O 

1 0 Higher levels of the memory hierarchy, / greater than 1 , are processed by a loop comprising 

1 1 = decision block 650 and process blocks 655, 660, 665, and 670. Decision block 650 begins the loop 

1 2 if for each level / of the memory hierarchy. As the loop begins, decision block 650 determines if level / 

1 3 is less than or equal to a maximum number of levels. If so, then process block 655 increments level /, 

14 M and then process block 660 sets n, equal to the size of the memory hierarchy for level i, i.e., to the 

1 5 ! page size in bytes for the page level of the memory hierarchy. Thereafter, process block 665 applies a 

16 - heuristic H to partition G, into clusters having sizes no greater than N, = n/N,., to produce 

17 C3 Clmt(G„n/NJ. Process block 665 may perform this clustering in the same manner as process block 

18 635 or it may use a different partitioning heuristic. Thereafter, process block 670 creates the next 

1 9 level PEG G, + , from the level /' partitioned PEG ClustfG^n/NJ. In this graph G, +/ = (V i+i , E,J 

20 constructed by process block 670, each node in V l+I corresponds to a cluster in G h and each node in 

2 1 V t ; t has a weight of one. An edge e = (u, v) exists in £, + , if and only if both u and v are in V- h u and v 

22 correspond to different clusters, and there is some edge in E, from one of the nodes in the cluster 

23 represented by u to some node in the cluster represented by v, or visa versa. The weight of e is the 

24 sum of the weights of all edges in E, that have endpoints in the two clusters (m and v). Process block 

25 670 performs this PEG construction in the same manner as process block 645. Thereafter, processing 

26 loops back to decision block 650, the beginning of the loop, to process the next level i+J. 
27 

28 Returning now to decision block 650, if level / is not less than or equal to a maximum number 

29 of levels, then process block 675 reorders basic blocks in memory by grouping all of the nodes of a 
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1 cluster in an adjacent order; process block 680 adds NOPs to fill out the unused portion of memory 

2 for that memory hierarchy level for each level of the memory hierarchy; process block 685 stores the 

3 reordered and padded basic blocks in memory beginning with the first memory location after a 

4 boundary for all the memory levels; and then the process ends at process block 690. 
5 

6 Referring now to Figure 7, an expansion of process block 670 is illustrated After process 

7 block 665 partitions the PEG, then process block 710, process block 720, process block 730, and 

8 process block 740 construct a Program Execution Graph G, +/ for the next higher level i + 1 wherein 

9 the nodes N gH of G, +/ are constructed from the clusters ClustfG^n/NiJ of G,. Process block 720 

1 0 ? assigns a weight of one to each node in Process block 730 assigns edges E i+t in G i+/ by placing 

1 1 EC an edge (u,v) between u and v, for each u and v of N^ h if and only if there is an edge between a pair 

1 2 of nodes in G„ one of which is in the cluster corresponding to u and one of which is in cluster 

1 3 y corresponding to v. Process block 740 assigns weights to the edges in G /+/ ; the weight of an 

14 edge (w,v) in G /+/ is the sum of the weights of the edges between component nodes in G / of the 

1 5 r clusterSj corresponding to u and v. Thereafter, processing loops back to decision block 650, the 

16 beginning of the loop, to process the next level /+/. 

17 g 

1 8 In an alternative embodiment of the present invention, only a single loop may be used for 

19 processing all levels of the memory hierarchy. In this alternative embodiment, process block 665 may 

20 be modified to include the functionality of process block 635 so that process block 665 partitions all 

2 1 PEG's. Process block 670 may also be modified to include the functionality of process block 645 so 

22 that it creates the next level PEG from all partitioned PEG's. Process block 640 may be moved within 

23 the loop to pad the first or additional level clusters. 
24 

25 After the clustering is completed, the basic blocks are reordered in memory by grouping all of 

26 the nodes of a cluster in an adjacent order (process block 675), although the ordering of the basic 

27 blocks within this adjacent grouping may be varied. Because clusters must not cross boundaries of 

28 memory hierarchies, NOPs are added to fill out the portion of a memory hierarchy level that is not 

29 filled by the clusters. For memory hierarchy level 1, NOPs are added to the end of any clusters that 
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1 have size less than some multiple of the cache line size so that the amount of memory required for 

2 each cluster is some integral multiple of the cache line size. For each subsequent level of the memory 

3 hierarchy, the minimum number of NOPs necessary to fill out the unused portion of memory for that 

4 memory hierarchy level are added (process block 680). After the processing for the last memory 

5 hierarchy is completed yielding the final PEG, the reordered basic blocks are stored in memory 

6 starting at a boundary of all the memory levels so that all of the basic blocks from one cluster are 

7 adjacent, i.e. interleaving from different clusters is not allowed (process block 685). This is true for 

8 every level of the hierarchy. 

9 i 

1 0 Referring now to Figure 8, a block diagram illustrates a computer system 800 used in 

1 1 performing the method of the present invention, forming part of the apparatus of the present 

1 2 3 invention, and which may use the article of manufacture comprising a computer-readable storage 

1 3 medium having a computer program embodied in said medium which may cause the computer system 

14 12 to practice the present invention. The computer system 800 includes a processor 802, which includes 

1 5 ^ a central processing unit (CPU) 804, a memory 806, and a multi-level memory cache 810 (level 1 

16 G cache (LI) 812 through level n cache (Ln) 814). Additional memory, in the form of a hard disk file 

1 7 storage 816 and a computer-readable storage device 818, is connected to the processor 802. 

1 8 Computer-readable storage device 818 receives a computer-readable storage medium 820 having a 

1 9 computer program embodied in said medium which may cause the computer system to implement the 

20 present invention in the computer system 800. The computer system 800 includes user interface 

21 hardware, including a mouse 822 and a keyboard 824 for allowing user input to the processor 802 

22 and a display 826 for presenting visual data to the user. The computer system may also include a 

23 printer 828. 
24 

25 Although the present invention has been particularly shown and described with reference to a 

26 preferred embodiment, it will be understood by those skilled in the art that various changes in form 

27 and detail may be made without departing from the spirit and the scope of the invention. 
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CLAIMS 

We claim: 

1 1 . A method of restructuring a program comprising basic blocks for execution by a processor 

2 having a memory hierarchy comprising a plurality of levels, said method comprising the steps 

3 of: 

4 i a ) constructing a Program Execution Graph (PEG) from control flow and 

5 frequency information from a profile of the program, the PEG comprising a weighted 

6 undirected graph comprising nodes representing the basic blocks and edges representing 

7 - transfer of control between pairs of the basic blocks, each of the nodes having a weight equal 

8 to the size of the basic block represented by the node, each of the edges having a weight 

9 equal to a frequency of transition between a pair of basic blocks represented by a pair of nodes 
1 0 connected by the edge; 



1 1 b) partitioning the nodes of the PEG into clusters such that a sum of weights of 

1 2 the edges whose endpoints are in different clusters is minimized, and such that for any cluster, 

1 3 a sum of weights of the nodes in the cluster is no greater than an upper bound; and 

14 c) restructuring the basic blocks into contiguous code corresponding tothe 

1 5 clusters. 
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1 2. The method of claim 1 further comprising the steps of: 

2 d) constructing a next PEG from the clusters of the partitioned PEG such that a 

3 node in the next PEG corresponds to a cluster in the partitioned PEG, and such that there is 

4 an edge between two nodes in the next PEG if there is an edge between components of the 

5 clusters represented by the two nodes; and 

6 e) assigning a weight to each node of the next PEG; and 

7 f) assigning a weight to an edge between a pair of nodes of the next PEG 

8 representing a pair of clusters of the partitioned PEG, the edge weight being a summation of 

9 weights of edges in the partitioned PEG having endpoints in the pair of clusters in the 
10 partitioned PEG. 

1 2 3 . The method of claim 2 further comprising the step of 

2 ' f) repeating steps b through f 

1 4. The method of claim 1 wherein the upper bound is a multiple of a size of a level of the 

2 memory hierarchy. 

1 5 . The method of claim 3 wherein the upper bound for a level of the memory hierarchy, other 

2 than a first level, is a size of the memory hierarchy level divided by an upper bound used to 

3 partition a next lower level of the memory hierarchy. 
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6. The method of claim 3 further comprising the steps of: 

removing a basic block whose size is greater than the upper bound from the 
partitioning step; and 

reintegrating the basic block whose size is greater than the upper bound into a next 
repetition of steps b through f. 
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An article of manufacture for use in a computer system for restructuring a program 
comprising basic blocks for execution by a processor having a memory hierarchy comprising a 
plurality of levels, said article of manufacture comprising a computer-readable storage medium 
having a computer program embodied in said medium which may cause the computer system 
to: 

a) construct a Program Execution Graph (PEG) from control flow and frequency 
information from a profile of the program, the PEG comprising a weighted undirected graph 
comprising nodes representing the basic blocks and edges representing transfer of control 
between pairs of the basic blocks, each of the nodes having a weight equal to the size of the 
basic block represented by the node, each of the edges having a weight equal to a frequency 
of transition between a pair of basic blocks represented by a pair of nodes connected by the 
edge; 

b) partition the nodes of the PEG into clusters such that a sum of weights of the 
edges whose endpoints are in different clusters is minimized, and such that for any cluster, a 
sum of weights of the nodes in the cluster is no greater than an upper bound; and 

c) restructure the basic blocks into contiguous code corresponding to the clusters. 
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1 8. The article of manufacture of claim 7 wherein the computer program may further cause the 

2 computer system to: 



3 d) construct a next PEG from the clusters of the partitioned PEG such that a node 

4 in the next PEG corresponds to a cluster in the partitioned PEG, and such that there is an 

5 edge between two nodes in the next PEG if there is an edge between components of the 

6 clusters represented by the two nodes; and 



7 S e) assign a weight to each node of the next PEG; and 



g f) assign a weight to an edge between a pair of nodes of the next PEG 

9 representing a pair of clusters of the partitioned PEG, the edge weight being a summation of 

10 weights of edges in the partitioned PEG having endpoints in the pair of clusters in the 

] 1 partitioned PEG. 



1 9. The article of manufacture of claim 8 wherein the computer program may further cause the 

2 computer system to: 

3 c f) repeat steps b through f. 

1 1 0. The article of manufacture of claim 7 wherein the upper bound is a multiple of a size of a level 

2 of the memory hierarchy. 

1 11. The article of manufacture of claim 10 wherein the upper bound for a level of the memory 

2 hierarchy, other than a first level, is a size of the memory hierarchy level divided by an upper 

3 bound used to partition a next lower level of the memory hierarchy. 
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1 12. The article of manufacture of claim 9 wherein the computer program may further cause the 

2 computer system to: 

3 remove a basic block whose size is greater than the upper bound from the partitioning 

4 step; and 

5 reintegrate the basic block whose size is greater than the upper bound into a next 

6 repetition of steps b through f. 
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1 13. A computer system for restructuring a program comprising basic blocks for execution by a 

2 processor having a memory hierarchy comprising a plurality of levels, said computer system 

3 comprising: 

4 a) a Program Execution Graph (PEG) constructed from control flow and 

5 frequency information from a profile of the program, the PEG comprising a weighted 

6 undirected graph comprising nodes/ representing the basic blocks and edges representing 

7 transfer of control between pairs of the basic blocks, each of the nodes having a weight equal 

8 to the size of the basic block represented by the node, each of the edges having a weight 

9 equal to a frequency of transition between a pair of basic blocks represented by a pair of nodes 
1 0 connected by the edge; 

u b) a partition of the nodes of the PEG into clusters such that a sum of weights of 

1 2 the edges whose endpoints are in different clusters is minimized, and such that for any cluster, 

1 3 I a sum of weights of the nodes in the cluster is no greater than an upper bound; and 



14 □ c) a restructuring of the basic blocks into contiguous code corresponding to the 

1 5 clusters. 
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1 14. The computer system of claim 13 further comprising: 

2 d) a next PEG constructed from the clusters of the partitioned PEG such that a 

3 node in the next PEG corresponds to a cluster in the partitioned PEG, and such that there is 

4 an edge between two nodes in the next PEG if there is an edge between components of the 

5 clusters represented by the two nodes; and 

6 e) a weight assigned to each node of the next PEG; and 

7 f) a weight assigned to an edge between a pair of nodes of the next PEG 

8 representing a pair of clusters of the partitioned PEG, the edge weight being a summation of 

9 weights of edges in the partitioned PEG having endpoints in the pair of clusters in the 
10 partitioned PEG. 



1 1 5 . The computer system of claim 1 4 further comprising: 

2 f) a repetition of elements b through f. 

1 16.. The computer system of claim 1 3 wherein the upper bound is a multiple of a size ofa level of 

2 the memory hierarchy. 

1 17. The computer system of claim 1 5 wherein the upper bound for a level of the memory 

2 hierarchy, other than a first level, is a size of the memory hierarchy level divided by an upper 

3 bound used to partition a next lower level of the memory hierarchy. 
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1 1 8. The computer system of claim 15 further comprising: 

2 a removal of a basic block whose size is greater than the upper bound from the 

3 partitioning step; and 

4 a reintegration of the basic block whose size is greater than the upper bound into a 

5 next repetition of elements b through f. 
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I ABSTRACT OF THE DISCLOSURE 

2 

3 A METHOD OF, SYSTEM FOR, AND COMPUTER PROGRAM PRODUCT FOR 

4 PROVIDING EFFICIENT UTILIZATION OF MEMORY HIERARCHY 

5 THROUGH CODE RESTRUCTURING 

6 

7 Code restructuring or reordering based on profiling information and memory hierarchy is 

8 provided by constructing a Program Execution Graph (PEG) corresponding to a level of the memory 



9 m hierarchy, partitioning this PEG to reduce estimated memory overhead costs below an upper bound, 

1 0 ^ and constructing a PEG for a next level of the memory hierarchy from the partitioned PEG. The PEG 

< « 

1 1 CO is constructed from control flow and frequency information from a profile of the program to be 

1 2 restructured. The PEG is a weighted undirected graph comprising nodes representing basic blocks 

1 3 * y and edges representing transfer of control between pairs of basic blocks. The weight of a node is the 

14 * size of the basic block it represents and the weight of an edge is the frequency of transition between 

15 q the pair of basic blocks it connects. The nodes of the PEG are partitioned or clustered into clusters 

16 JZ such that the sum of the weights of the nodes in any cluster is no greater than an upper bound. A 

1 7 O next PEG is then constructed from the clusters of the partitioned PEG such that a node in the next 

18 PEG corresponds to a cluster in the partitioned PEG, and such that there is an edge between two 



19 nodes in the next PEG if there is an edge between the clusters represented by the two nodes. Weights 

20 are assigned to the nodes and edges of the next PEG to produce a PEG, and then the PEG 

21 partitioning, basic block reordering, and PEG construction steps may be repeated for each level of the 

22 memory hierarchy. After the clustering is completed, the basic blocks are reordered in memory by 

23 grouping all of the nodes of a cluster in an adjacent order beginning at a boundary for all the levels of 

24 the memory hierarchy. Because clusters must not cross boundaries of memory hierarchies, NOPs are 

25 added to fill out the portion of a memory hierarchy level that is not filled by the clusters. 
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